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1 . Abstract 

Performance Seeking Control (PSC) attempts to find 
and control a process at an operating condition that will 
generate maximum performance. In this paper a nonlinear 
multivariable PSC methodology will be developed, utiliz- 
ing the Fuzzy Model Reference Learning Control 
(FMRLC) and the method of Steepest Descent or Gradient 
(SDG). This PSC methodology employs the SDG method 
to find the operating condition that will generate maxi- 
mum performance. This operating condition is in turn 
passed to the FMRLC controller as a set point for the 
control of the process. The conventional SDG algorithm is 
modified in this paper in order for convergence to occur 
monotonically. For the FMRLC control, the conventional 
fuzzy model reference learning control methodology is 
utilized, with guidelines generated here for effective tun- 
ing of the FMRLC controller. 

2. Introduction 


PSC approaches, primarily developed for aircraft 
applications, are largely based on dynamic or linear pro- 
gramming with some limited work having been per- 
formed on gradient-type PSC. Even though random-type 
search techniques for PSC (like linear or dynamic pro- 
gramming) have proven effective for various applica- 
tions/ 1 " 3 ) a number of difficulties can be encountered in 
the implementation of these techniques such as poor 
convergence or long execution times. 

In this paper, the combined effect of gradient optimi- 
zation and FMRLC control with its ability to perform 
nonlinear control, with fast on-line learning of the control 
law, will be exploited. The state trajectory that generates 
maximum performance resulting from the SDG 
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algorithm, is passed to the FMRLC controller as the 
desired set point for the control of the process. Fig. 1 . The 
computed control trajectory can also be utilized by the 
FMRLC, if it is desired to use the FMRLC as a trim 
controller. 

The standard SDG control algorithm can fail to con- 
verge if the initial guess of the control inputs is rather poor. 
Therefore the algorithm is modified here in order for con- 
vergence to occur monotonically, which makes the algo- 
rithm more suitable for on-line implementation. 

During the past several years, fuzzy control has 
emerged as one of the most active and promising control 
areas, especially because of the ability of fuzzy control in 
controlling highly nonlinear, time variant, and ill-defined 
systems. The works of Mamdani and his colleagues on 
fuzzy control 111 ' 14 ) was motivated by Zadeh’s work on 
the theory of fuzzy sets/ 16 ' 19 ) and its application to lin- 
guistics and systems analysis. The work of Procyk and 
Mamdami on the linguistic self-organizing controller 120 ) 
as well as refinements to this algorithm made by others, 
was later modified and extended by Layne to what it is 
called FMRLC control/ 21 ) The FMRLC structure, Fig. 4, 
has learning capabilities and differs conceptually from 
adaptive control primarily by its ability to memorize 
learned experiences. The FMRLC algorithm will be uti- 
lized here for nonlinear, multivariable feedback control, 
while some guidelines will be generated for the effective 
tuning of the FMRLC controller. In this paper the SDG 
and FMRLC controllers will be combined to form the new 
PSC control structure shown in Fig. 1. 

It is assumed here that the model of the plant is 
available. However, a standard Kalman-filter estimator, 
or some other identification technique, could be used to 
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Figure 1 . — FMRLC/SDG PSC Control structure. 


x — a(x(t),u(t),t) (3) 

The performance index selected for this problem has 
the form 

/[x(0, tf ] = | [x (; tf Qx(t ) + u(tf Ru(t)\flt (4) 

0 

where: x = [(xj - x 10 )(x 2 -x 20 )]/ m = [u^f. 


n oi 

, and R = 

'0.5 

0 ' 

.0 i 


0 
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identify the plant on-line if necessary/ 8,9 ) The implied 
assumption here is made that the solution space is convex, 
in order for the SDG control to find the global extremal 
within the constraints of the control inputs. 

A nonlinear process is selected to demonstrate the 
effectiveness of this control methodology. The process in 
(1) is chosen to be nonlinear, stable, with significant cross 
coupling of the control inputs to the controlled variables. 


and Xjq, x 2 () are constants. 

For the case where h(x(tj),tf ) = 0 in (2), the hamiltonian 
of the system in Eq. (3) with the performance index (2) can 
be expressed as: 

H(x(t),u(t),p(t)) = g(x(t),u(0>t) + p(t)[a(x(t),u(t),t] 

(5) 


Xi — —2 x}X 2 + 3^2 + u l 

( 1 ) 

2 3 , 

X 2 = Xi — X2 + «2 

The process itself (i.e. with zero control input) is deter- 
mined to be stable by using the Liapunov Direct method, 
with the Liapunov function: V(Xj,x 2 ) = ax* m + bx^ n . With 
the choices of m = 1, n = 1, a = 1, b = 2, which 
simplifies V(x),V(x v x 2 ) = x 2 + 2x 2 2 , which is positive 
definite. V(x) = W(x(t)f f(x(t)) , and with no control 
input, V (x t ,x 2 ) = -4x 2 4 + 6x { x^ which is negative 
semidefmite as long as the inequality 2x 2 > 3Xj is satisfied. 

In section 3, the formulation of SDG optimization for 
the process in Eq. ( 1 ) and the performance index in Eq. (2) 
will be carried out. In section 4 the FMRLC control 
methodology will be discussed and the multivariable 
control design for the process in Eq. (1) will be shown. 
Section 5 will cover the conclusion. 


where p{t) are the costate equations. Substituting into 
Eq. (5) the functions g and a from Eqs. (2) and (3) relative 
to the plant in Eq. ( 1 ) and the performance index in Eq. (4), 


H = (x 1 - x lo f + (x 2 - x 2o f + 0 .5uf + 0.5wf 

+ p x (-2x 1 x 2 +3 x 2 +ui) + p 2 [x\ ~ x 2 + u 2 ) (6) 

The costate equations are: 

/>!=-— = — 2(*1 -x l0 ) + 2 Pl x 2 - 2p 2 x x 
ox i 

(7) 

P2 = — = ~ 2 { x 2 ~ x 2o ) + P\( 2x l - 3) + 3 p 2 x\ 

dx 2 


The partials of the hamiltonian with respect to the 
controls are: 


3.1 Steepest Descent Gradient Formulation 

Generally a performance index and the plant states 
can be expressed as: 

f f 

J = /i|x(t / \tf ) + | «(*)> *)dt (2) 

^0 


a h 

— - Ml + P\ 

OUl 

(B) 

dH 

- — -U 2 +P 2 

du 2 


The control history t e is used to solve 
the differential equations in (3) and (7) (including the 
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superscripts x^, u^\ p® in (3) and (7), and (i) is an index 
signifying the current iteration), so that the nominal state- 
costate trajectory satisfies the boundary conditions: 


ate dH^Idu, t e [t () ,tj] and store in the computer. 

The following steps from 3 . 1 to 3 .2 constitute changes 
to the SDG algorithm to achieve monotonic convergence: 


The initial conditions in (9) are split boundary, and 
the differential equation in (3) is solved forward in time, 
where Eq. (7) is solved backwards in time. Since h(x(tj-),tj) 
= 0 in Eq. (2), = 0. If this nominal control history 

also satisfies 


— (^'■>(0,« <i) (0,/ ) (0,() = o, (10) 

which is equal to Eq. (8) for this problem, then 
are extremals. If Eq. (10) is not satisfied, successive con- 
trol histories are adjusted as follows: 

(t k ) = « (i) ((* ) - T (t k ) k = 0,...,N-\ 

( 11 ) 

where, x > 0,is selected for the desired effect dH/du Eq. (8) 
will have on the new control, and t k signifies a discrete 
time. Note that the elements of Eq. (11) signify vector 
quantities for a multivariable control process. The itera- 
tive computation process is terminated when norm two, 


dH_ 
du 2 



a h 

du 


(0 


; dt 


at iteration (i) is less 


3.1 Evaluate the performance index /(/). If J (i> > J iiA> 
restore the previous control trajectory (i.e. set u (i) (t) 
= t € [t 0 ,t^) and repeat step 2 and 3 from above. 


3.2 Find the component of 


dH^_ 

du 


with the largest 


magnitude and decrease its corresponding x component 
(halving x is found to work fine for various problems). 

3H (i) II 

4. If < Tj or l/° - 7 (i ' 1) l < y 2 then terminate the 


iterative procedure and output the extremal states and 
control. If the stopping criterion is not satisfied, keep in 
storage the previous control vector (this step is added 
relative to changes for monotonic convergence), and 
generate a new piecewise-constant control history given 
by Eq. (11). Return to step 2. 


3.2 Steepest Descent Gradient Simulation 


The final time, tj, in Eq. (4) can be chosen appropri- 
ately based on the settling time of the process to a step 
input, Fig. 2. The fourth order Runge-Katta integration 
method was used for forward and backwards integration 
of Eqs. (1) and (7) respectively. Euler integration was also 
used as an alternative, but no noticeable differences in the 
simulation results were observed for this problem. Choos- 
ing rather large x's in Eq. (11), would normally cause the 
steepest descent algorithm to diverge or fail to converge. 
However, the modifications made to the algorithm in this 
paper will cause the algorithm to monotonically converge, 
because x is adjusted automatically. Figure 3 shows the 
monotonically decreasing performance index, the state 
trajectories for the choice of (x 10 ,x 20 ) = (1 .3, 1 .2), and the 
resulting control trajectories. 


than a preselected positive termination constant y l5 or the 
performance index 17°) - ^ y 2 . The basic SDG 

algorithm listed below has been modified as to achieve 
monotonic convergence. 

SDG Algorithm Steps: 

1. Select a discrete piecewise-constant approxima- 
tion to the nominal control history u^°\t), t e [t 0 ,t^. 

2. Using the nominal control history u^\ integrate 
the state equations from to tj with initial conditions 
x(t 0 ) = x 0 and store the resulting trajectory. 

3. Integrate the costate equations backwards in time 
from tj- to t Q using p(tj) = joy as the initial condition and the 
piecewise-constant values of x^. At each time step evalu- 


4.1 Fuzzy Model Reference Learning Control 

Fuzzy control theory will not be covered in depth in 
this paper. For more detail discussions in these areas see 
Refs. 1 1 to 22. The FMRLC structure/ 20 ) shown in Fig. 4, 
employs an inverse fuzzy model of the process and modi- 
fies the knowledge base through the knowledge base 
modifier mechanism in order for the process output y{kt) 
to match the reference model output ym(kt). In this section 
the basic design procedure of the FMRLC for the process 
in Eq. (1) will be discussed. 

For the MIMO system discussed in this paper two 
decoupled FMRLC controllers are constructed. A coupled 
FMRLC controller could be utilized instead, however, the 
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Figure 2. — Process open loop step response. 


dimensions of the knowledge bases would have increased 
equivalent to the number of the inputs to the fuzzy con- 
troller. In addition to the basic FMRLC structure shown 
in Fig. 4, a pole at zero frequency was placed at the output 
of each decoupled controller. This is needed for zero 
steady state error. Each decoupled FMRLC controller 
contains 6 adjustable gains. Therefore, some discussion 
in this section will be devoted to establishing some 
guidelines for the effective tuning of the control gains. 
Typical inputs to the fuzzy controller are the error e(kT) 
and the error derivative c(kT), but other types of inputs 
can be chosen such as integration of the error. The 
membership functions for all the inputs to the fuzzy 
controllers and the inverse models have been chosen with 
triangular shape, normalized, and uniformly distributed 
in each Universe of Discourse, as shown in Fig. 5. In 
Fig. 5, E-> signifies a membership function or linguistic 
value associated with a specific input to the fuzzy control- 
ler, where [i gives the certainty that an element of that 
particular input may be classified heuristically as EE 
Figure 6 shows the rule base constructed for the inverse 
fuzzy models. From this rule base it can be deduced that 




Time, sec 



Figure 3. — Steepest descent optimiation. 


the Consequent membership functions corresponding to 
the inverse model output variable y^(kT) have similar 
distribution to the membership functions shown in Fig. 5. 
The knowledge base (rule base) contains the centers of 
the membership functions which are triangular shaped for 
this problem, with a base width of 0.4 as seen in Fig. 5. 
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E-5 E" 4 E-3 E" 2 E" 1 



- 1 . 0 - 0 . 8 - 0 . 6 - 0 . 4 - 0.2 0.0 + 0 . 2 + 0 . 4 + 0 . 6 + 0 . 8 + 1 .0 
Figure 5. — Membership functions . 
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Figure 6. — Inverse fuzzy model rule base . 
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One of the important consideration in the construction of 
the inverse knowledge base is that the inverse fuzzy model 
exhibits the proper directionality associated with the 
controlled process. The knowledge base associated with 
the fuzzy controllers initially contains all zeros, which 
reflects no knowledge of how to control the process. This 
knowledge base is updated auto-matically as the FMRLC 
controller learns how to control the process. 


The selection of the FMRLC gains is an important 
step in the design process, as the ability of the controller 
to track the reference model will heavily depend on the 
particular choices of the gains. The gains g e and g y are 
chosen so that the ranges of these inputs are mapped to 
a normalized universe of discourse in the range of [-1,1]. 
For instance an appropriate choice for the value of the gain 
g e would be H range ( e(kt )). A good choice for the value 
of the gain g c is found to be approximately equal to 
10 /{range (e(&f))/7), which is equal to 10/(max change 
(r(kt))/T), where r(kt) is the set point and T is the sampling 
time. The smaller the choice for the values of the gains g e 
and g c , the more the control action is concentrated towards 
the center region of the rule base, resulting in better control 
tracking at the expense of an increased control rate of the 
control variable u(kT). The gain, g y , effects the damping 
of the process response: If it’ s too small the response will 
be oscillatory, if it’s too large, the process will be unable 
to keep up with the reference model. A good choice for the 
value of the gain, g y , is found to lie somewhere in the 
range of [1/(4 co n ), l c /(2co n )], where, co n , is the natural 
frequency of the process. The output gains, g u and gp are 
chosen so that the corresponding Normalized Universe of 
Discourse maps to the range of the output variables of the 
fuzzy controller. For instance, both g u and gy- are selected 
to be equal to the range of the control input variable, u(kT). 
This choice for the output gains allows both u(kT) and 
yfltT) to take on values as large as the largest control input. 


The selection of the reference model shown in Fig. 4, 
represents the desired performance of the FMRLC feed- 
back control system. The reference model is selected here 
to have a natural frequency, co m , equal to the process 
natural frequency, co n , with a relatively low step value for 
the open loop response. With the process being nonlinear, 
its response time can strongly depend on the magnitude of 
the control input. Therefore, it may not be desirable to 
select a reference model significantly faster than the 
process response time relative to a low control input value, 
or else we may be asking for relatively large control rates. 
A first order model for the selection of the reference model 
has been found to be adequate. 


g rm ~ 


S + (O m 


( 12 ) 


4.2 FMRLC Control Simulation 

Based on the discussion in section 4.1, the control 
parameters for the two decoupled FMRLC controllers 
have been selected with the following values: 

Sel <?cl £y c l 8ul Sf\ ®/nl 

8e 1 Sy e 2 Sc2 8y c 2 Su2 8f2 CO m2 

'0.25 0.25 0.1 0.125 10 10 2' 

0.25 0.25 0.1 0.125 10 10 2 


The defuzzification approach used in this simulation 
is the so called “Center of Gravity.” Figure 7 shows the 
response of the decoupled FMRLC controller with simul- 
taneous step set point changes. This response shows the 
tracking capabilities of the FMRLC. The set point track- 
ing response was used to tune the controller as was 
discussed in section 4. 1 . The knowledge base of the fuzzy 




Figure 7. — FMRLC close loop step response. 
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Figure 8.— Automatically generated rule base for State X 2 . 


controller started with all zero entries, reflecting that 
initially there was no knowledge of how to control the 
system. The learning rate is quite fast as is evident from the 
responses of the states and control inputs in Fig. 7. The 
resulting knowledge base of the decoupled controller 
corresponding to the state x 2 , for the simulation in Fig. 7, 
is shown in Fig. 8. The zero elements associated with this 
knowledge base is an indication that the controller, for 
this particular simulation, has not had the opportunity to 
venture in these areas of its knowledge space. 

For a complete simulation of the PSC approach 
discussed in this paper; in section 3 with the SDG formula- 
tion and the modified SDG algorithm, the optimal state 
and control trajectories were found, Fig. 3. The states 
trajectories are passed to the FMRLC controller as the 
desired set point control for the control of the process in 
Eq. (1), shown in Fig. 9. In the simulation (Fig. 10), the 
control trajectories were also utilized by adding them to 
the control output of each decoupled FMRLC controller as 
you would for a trim control structure. Due to memory 
limitations for the PC Fortran used in this simulation, the 
state and control trajectories of Fig. 3 were approximated 
by first order responses, which would be expected to 
introduce error in the trim control. All the control para- 
meters discussed in this section remained the same for the 
simulations shown in Figs. 9 and 10, except for the refer- 
ence model frequency, co m , which was increased to 
16 rad/sec. For these simulations, the knowledge bases of 
the two decoupled controllers were also initialized with 
zeros, but the resulting knowledge bases from the simula- 
tion in Fig. 7 could have been used as the starting point. 



Time, sec 



Figure 9. — FMRLC simulation without trim control. 


NASA TM-1 07454 


7 












Figure 10. — FMRLC simulation with trim control. 


5.0 Conclusion 

In this paper a nonlinear process was used to help 
develop a PSC methodology that utilized the modified 
SDG and FMRLC approaches. The simulation results 
presented in this paper showed that the modified SDG 
algorithm can be used effectively to compute off line the 
optimal state and control trajectories for the control pro- 
cess. The results also showed that the FMRLC approach 
with the given tuning guidelines, can be used to control the 
nonlinear multivariable control process with good track- 
ing performance, and relatively fast on line learning of the 
control law. Finally, the optimal states and control trajec- 
tories computed by the SDG algorithm are utilized in this 
paper by the FMRLC to control the process to achieve the 
desired performance. 

For future work it would be important to study stabil- 
ity, convergence, and robustness of this approach in more 
detail. Further, experimental validation of this method 
would be needed, with processes that exhibit more 


complex dynamics. Finally, a direct comparison with other 
existing PSC control methodologies could be carried out. 
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